full-FORCE: A Target-Based Method for Training Recurrent Networks

Trained recurrent networks are powerful tools for modeling dynamic neural computations. We present a target-based method for modifying the full connectivity matrix of a recurrent network to train it to perform tasks involving temporally complex input/output transformations. The method introduces a second network during training to provide suitable "target" dynamics useful for performing the task. Because it exploits the full recurrent connectivity, the method produces networks that perform tasks with fewer neurons and greater noise robustness than traditional least-squares (FORCE) approaches. In addition, we show how introducing additional input signals into the target-generating network, which act as task hints, greatly extends the range of tasks that can be learned and provides control over the complexity and nature of the dynamics of the trained, task-performing network.Comment: 20 pages, 8 figure

Remembrance of things practiced with fast and slow learning in cortical and subcortical pathways

13 pagesThe learning of motor skills unfolds over multiple timescales, with rapid initial gains in performance followed by a longer period in which the behavior becomes more refined, habitual, and automatized. While recent lesion and inactivation experiments have provided hints about how various brain areas might contribute to such learning, their precise roles and the neural mechanisms underlying them are not well understood. In this work, we propose neural- and circuit-level mechanisms by which motor cortex, thalamus, and striatum support motor learning. In this model, the combination of fast cortical learning and slow subcortical learning gives rise to a covert learning process through which control of behavior is gradually transferred from cortical to subcortical circuits, while protecting learned behaviors that are practiced repeatedly against overwriting by future learning. Together, these results point to a new computational role for thalamus in motor learning and, more broadly, provide a framework for understanding the neural basis of habit formation and the automatization of behavior through practice

Performance results for networks trained on the delayed comparison task.

<p>(a & b) <i>f</i>_in(<i>t</i>) (grey), <i>f</i>_hint(<i>t</i>) (grey dotted), <i>f</i>_out(<i>t</i>) (black dotted) and <i>z</i>(<i>t</i>) (orange) for a network of 1000 units. Networks trained with full-FORCE learning without (a) and with (b) hints. Three different trials are shown from bottom to top: an easy “ + ” trial, a difficult “-” trial, and an easy “-” trial. (c & d) Test performance for networks trained without (c) and with (d) a hint. Each dot indicates a test trial and the dot color indicates the reported output class (“ + ” cyan or “-” red). The horizontal axis is the interpulse delay and the yellow region indicates the training domain. The vertical axis indicates the pulse amplitude difference.</p

Test error as a function of number of units.

<p>Normalized test error following network training for full-FORCE (a) and FORCE (b) as a function of network size. Each dot represents the test error for one random initialization of <b>J</b>^D. Test error was computed for 100 random initializations of <b>J</b>^D for each value of <i>N</i>. The line indicates the median value across all simulations, and the size of each dot is proportional to the difference of that point from the median value for the specified network size.</p

Testing does not improve with more training.

<p>Median test error for full-FORCE (a) and FORCE (b) computed across 200 random initializations of <b>J</b>^D for networks trained on the oscillation task. Three different size networks are shown, 100, 200 and 400 units, where larger networks correspond to lighter colors. The horizontal axis shows the number of batches used to train the network, where each batch corresponds to 100 oscillation periods.</p

Network architecture.

<p>(a) Task-performing network. The network receives <i>f</i>_in(<i>t</i>) as an input. Training modifies the elements of <b>J</b> and <b>w</b> so that the network output <i>z</i>(<i>t</i>) matches a desired target output function <i>f</i>_out(<i>t</i>). (b) Target-generating network. The network receives <i>f</i>_out(<i>t</i>) and <i>f</i>_in(<i>t</i>) as inputs. Input connections <b>u</b>, <b>u</b>_in and recurrent connections <b>J</b>^D are fixed and random. To verify that the dynamics of the target-generating network are sufficient for performing the task, an optional linear projection of the activity, <i>z</i>^<i>D</i>(<i>t</i>), can be constructed by learning output weights <b>w</b>^D, but this is a check, not an essential step in the algorithm.</p

Test error as a function of number of units.

<p>Normalized test error following network training for full-FORCE (a) and FORCE (b) as a function of network size. Each dot represents the test error for one random initialization of <b>J</b>^D. Test error was computed for 100 random initializations of <b>J</b>^D for each value of <i>N</i>. The line indicates the median value across all simulations, and the size of each dot is proportional to the difference of that point from the median value for the specified network size.</p

Performance results for networks trained on the interval timing task.

<p>(a & b) <i>f</i>_in(<i>t</i>) (grey), <i>f</i>_hint(<i>t</i>) (grey dotted), <i>f</i>_out(<i>t</i>) (black dotted) and <i>z</i>(<i>t</i>) (orange) for a network of 1000 units. Networks trained with full-FORCE learning without (a) and with (b) a hint for various interpulse intervals (100, 600, 1100, 1600 and 2100 ms from bottom to top). (c & d) Target response time plotted against the generated response time without (c) and with (d) hints. Each dot represents the timing of the peak of the network output response on a single test trial. Grey dots indicate that the network output did not meet the conditions to be considered a “correct” trial (see main text). Red dots show correct trials.</p